Holistically Constrained Local Model: Going Beyond Frontal Poses for Facial Landmark Detection

Facial landmark detection is an essential initial step for a number of facial analysis research areas such as expression analysis, 3D face modeling, facial attribute analysis, and person recognition. It is a well researched problem that has seen a surge of interest in the past couple of years. However, most state-of-the-art methods still struggle in the presence of extreme head pose, especially in challenging in-the-wild images. Furthermore, as most methods operate in a local manner [1, 2], they rely on good and consistent initialization, which is often very difficult to achieve. While some images attempt to combat this by evaluating a number of proposals and initializations, this comes at a computational cost. In our work, we present a new model – Holistically Constrained Local Model (HCLM), which unifies local and holistic facial landmark detection by integrating head pose estimation, sparse-holistic landmark detection and denselocal landmark detection. Our method’s main advantage is the ability to handle very large pose variations, including profile faces. Furthermore, our model integrates local and holistic facial landmark detectors in a joint framework, with a holistic approach narrowing down the search space for the local one. For a given set of k facial landmark positions x= {x1,x2, ...,xk}, our HCLM model defines the likelihood of the facial landmark positions conditioned on a set of sparse landmark positions Xs = {xs, s ∈ S} (|S| k) and image I as follows: